Robust Forecast Comparison*
نویسندگان
چکیده
Forecast accuracy is typically measured in terms of a given loss function. However, as a consequence of the use of misspecified models in multiple model comparisons, relative forecast rankings are loss function dependent. This paper addresses this issue by using a novel criterion for forecast evaluation which is based on the entire distribution of forecast errors. We introduce the concepts of general-loss (GL) forecast superiority and convex-loss (CL) forecast superiority, and we establish a mapping between GL (CL) superiority and first (second) order stochastic dominance. This allows us to develop a forecast evaluation procedure based on an out-of-sample generalization of the tests introduced by Linton, Maasoumi and Whang (2005). The asymptotic null distributions of our test statistics are nonstandard, and resampling procedures are used to obtain the critical values. Additionally, the tests are consistent and have nontrivial local power under a sequence of local alternatives. In addition to the stationary case, we outline theory extending our tests to the case of heterogeneity induced by distributional change over time. Monte Carlo simulations suggest that the tests perform reasonably well in finite samples; and an application to exchange rate data indicates that our tests can help identify superior forecasting models, regardless of loss function. JEL Classification: C12, C22.
منابع مشابه
A new metric for comparing precipitation patterns with an application to ensemble forecasts
[1] Ensemble forecasting can be seen as serving two purposes: (1) by comparison of the control and ensemble members to the observed precipitation field, one can assess the forecast performance probabilistically; and (2) by comparison of ensemble members to the control forecast, one can assess the ‘‘diversity’’ of an ensemble and quantify the uncertainty of the forecast. Both problems are ground...
متن کاملA Comparison Between Time Series, Exponential Smoothing, and Neural Network Methods To Forecast GDP of Iran
متن کامل
Computational aspects of robust Holt-Winters smoothing based on M-estimation
To obtain a robust version of exponential and Holt-Winters smoothing the idea of M -estimation can be used. The difficulty is the formulation of an easy-to-use recursive formula for its computation. A first attempt was made by Cipra (Robust exponential smoothing, J. Forecast. 11 (1992), 57–69). The recursive formulation presented there, however, is unstable. In this paper, a new recursive compu...
متن کاملThe Performance of Forecast-Based Monetary Policy Rules under Model Uncertainty∗ Andrew Levin Board of Governors of the Federal Reserve System
We investigate the performance of forecast-based monetary policy rules using five macroeconomic models that reflect a wide range of views on aggregate dynamics. We identify the key characteristics of rules that are robust to model uncertainty: such rules respond to the one-year-ahead inflation forecast and to the current output gap and incorporate a substantial degree of policy inertia. In cont...
متن کاملForecasting Gold Price using Data Mining Techniques by Considering New Factors
Gold price forecast is of great importance. Many models were presented by researchers to forecast gold price. It seems that although different models could forecast gold price under different conditions, the new factors affecting gold price forecast have a significant importance and effect on the increase of forecast accuracy. In this paper, different factors were studied in comparison to the p...
متن کامل